Automatic Utterance Segmentation in Spontaneous Speech

As applications incorporating speech recognition technology become widely used, it is desireable to have such systems interact naturally with its users. For such natural interaction to occur, recognition systems must be able to accurately detect when a speaker has finished speaking. This research presents an analysis combining lower and higher level cues to perform the utterance endpointing task. The analysis involves obtaining the optimal parameters for the signal level utterance segmenter, a component of the speech recognition system in the Cognitive Machines Group, and exploring the incorporation of pause duration and grammar information to the utterance segmentation task. As a result, we obtain an optimal set of parameters for the lower level utterance segmenter, and show that part-of-speech based N-gram language modeling of the spoken words in conjunction with pause duration can provide effective signals for utterance endpointing. Thesis Supervisor: Deb Roy Title: Assistant Professor of Media Arts and Sciences